Global Dynamics in Neural Networks II

Abst ract . Det ermining just what tasks are computable by neural networks is of fun damental importance in neural comp uting . The configuration space of several mod els of paral lel computation is essentially the Cantor middle-t hird set of real numbers. The HedlundRichardson th eorem states that a transformation from th e Cantor set to it self can be realized as th e global dynamics of a cellular automaton if and only if it takes th e quiescent configuration to itself , commut es wit h shifts , and is continuous in the product topology. An analogous theor em characterizing th e realizability of self-mappi ngs of the Cantor set as net -input global dynamics of neural netwo rks has recent ly been established . Here we give a char act erization of such realizability as the mor e na tural activa tion global dyn amics of neural networks. We also pr esent such a characte rization for realizability via global dyn amics of more general au tomata networks. This dynamical systems appr oach to neural computing allows pr ecise formulations of signifL cant problems about the computational power of neural networks.